Method and apparatus for utilizing representational images in commercial and other activities

ABSTRACT

In a method and apparatus for an object using a representational image, positioning the object at a predetermined position relative to the representational image, illuminating the representational image, presenting information from the representational image to an operator, at least one of controlling the illuminating of the representational image and presenting of information from the representational image to an operator, using a processing arrangement, and processing the object using the information obtained from the representational image.

CROSS REFERENCE TO RELATED APPLICATION

This application is a continuation of and claims priority to and thebenefit of U.S. patent application Ser. No. 12/235,579, filed on Sep.22, 2008, now U.S. Pat. No. 7,664,305 issued on Feb. 16, 2010, whichclaims priority to and is a continuation of U.S. patent application Ser.No. 11/856,059, filed on Sep. 17, 2007, now U.S. Pat. No. 7,428,327issued on Sep. 23, 2008, which claims priority to and is a divisional ofU.S. patent application Ser. No. 10/008,085, filed on Oct. 18, 2001, nowU.S. Pat. No. 7,286,696 issued on Oct. 23, 2007, which claims priorityto U.S. Provisional Patent Application Ser. No. 60/241,304 filed Oct.18, 2000, each of which is hereby expressly incorporated herein byreference in its entirety.

FIELD

The present invention relates to a method for creating and usingrepresentational images, including holographic representational imagesin various activities, including industrial processes, manufacturing andquality control/quality assurance.

BACKGROUND

Representational images are very useful as a tool for visual guidanceduring a wide range of activities involved in industry, including, forexample, manufacturing, product development, strategic planning, changeimplementation and verification and quality assurance and control.Representational images may range, for example, from schematic diagramsof electronic circuits or construction blueprints to fabric cuttingdiagrams, component layout and placement, and images on surfaces.

In the manufacturing sector, with the continual miniaturization andincreasing complexity of many types of products and with the concern forcost minimization during the manufacturing process, many companies haveconverted to automated assembly or robotics to replace their human workforce. Other companies have attempted to streamline and optimize themanufacturing process by hiring industrial engineers and/or consultantsto restructure their assembly lines. This has led to single taskassembly stations and additional employee training to increaseefficiency and to reduce costs, errors, worker fatigue and productassembly time. In both automated manufacturing and optimized humanassembly lines, however, schematic diagrams or other forms of printedinstructions or plans, are necessary to assist in the manufacturingprocess.

Despite the improvements by manufacturing companies, through the use ofautomation or assembly line optimization, there remains room foradditional improvement and further optimization of the manufacturingprocess. As described herein, further optimization and cost reductionsin both the automated and optimized human manufacturing lines throughthe use of representational images, including holographicrepresentational images, may be achieved.

For example, there are many products that require some hand assembly orhand wiring to complete their manufacture. In these instances, therepresentational image used during a manual assembly operation is aschematic diagram which guides the operator and which is used to confirmthe proper assembly operation for the workpiece. These schematicdiagrams must be placed within the visual range of the operator, and areusually posted directly in front of the operator. However, the positionof the schematic diagram requires the operator to look away from theobject being assembled to ascertain the proper step. After manyrepetitions of the same operation, operators usually commit the diagramto memory and rely on their memory to ensure the correctness of theiractivity, or they become familiar with the assembly step and no longerrequire a visual check of the schematic diagram to confirm properassembly. However, with long work shifts, occasional distractions, and avariety of other factors, including human error, a certain percentage ofthe assembly is performed incorrectly. It is only at the nextworkstation or in the QA/QC test area that the error may be discovered.The workpiece is then either discarded or repaired, both options addinga significant cost to the total manufacturing costs.

In low technology areas, such as the garment industry, mass productionof standardized or standard sized goods requires precision in thecutting of patterns. The cutting operations require the physicalplacement of a paper pattern on the fabric or material and the physicalcutting of the fabric or material by hand operation. Many companies haveswitched to automated cutting machines to increase efficiency, whileothers have moved the cutting operations overseas to reduce costs. Manytypes of garments, however, require certain hand operations from cuttingto assembly that either cannot be done by machine or if done by handrequire extensive operator training or have high levels of operatorerror.

In other areas such as detailed painting and decal or preprintedmaterial application, the operator must spend time prior to actuallyperforming the painting or application to first outline or mark theareas to be painted or where the materials are to be applied or thevarious patterns using a picture or detailed layout diagram. Only whenthe outlining or marking is complete can the operator fill in theoutlined areas with the proper colors or apply the materials.

These are just some of the many areas where schematic diagrams orrepresentational images are currently being used, and where improvementsare possible and feasible through the use of representational images,including holographic representational images, as described herein.

SUMMARY

The present document provides a method for creating and usingrepresentational images, including holographic representational images,and an apparatus for using representational images, to increaseefficiency, reduce costs, and provide greater amounts of information fora variety of tasks and analysis across many different sectors of theworkplace and the home.

The representational images described herein may be used as a Two orThree Dimensional Visual Template (2DVT or 3DVT) type ofrepresentational image presented in superimposition with the object tobe manufactured. The 2DVT and 3DVT superimpose the visual image of thetask specific information onto the workpiece, for example, the workpieceas it will appear on completion of that particular step in themanufacturing process, onto the workpiece from the previous step(s).Additionally, the 3DVT places the visual image in exact register,including depth, with the workpiece. The 2DVT utilizes the superimposedimage to capture in 2-D the visual image of its subject, and the 3DVTutilizes the unique properties of a hologram to capture in full 3-D,including depth, the visual image of its subject. The 2DVT and 3DVT willenable the worker to clearly see the precise location or path, forexample, of the component that is to be added to the object to bemanufactured at that point in the manufacturing process. The 2DVT and3DVT also allow for clear and effective visual emphasis or instructionregarding the step to be accomplished without the risk of translationalerrors, language impediments, learning curves or other problemsassociated with operator fatigue and training.

A representational image may be created by hand, using conventionalphotography or by computer aided design. These representational imagescan be superimposed onto a workpiece where a simple two-dimensionalrepresentational image (2DRI) is sufficient for the operation beingcontemplated, such as, for example, component assembly, painting, thecutting of fabric or the location of a bend or cut in a workpiece. Thesuperimposition of these representational images may be accomplished,using, for example, a slide or computer image projector, an overheadprojector, a high intensity computer display, an optical combiner, aprismatic screen, a splitter, or some other form of image projection ordisplay, including using a computer controlled display, such as, forexample, a liquid crystal display (LCD), a plasma display, a lightemitting diode (LED) display, spacial light modulator, fiber optics, orany other display that can either project an image onto or superimposean image with an object (actually or virtually) using a light or othervisible energy source, or allow light to pass through for projecting orsuperimposing an image of the displayed information directly orindirectly, thereby causing a superimposition of the image and theobject.

The representational images may also be created or prepared in onelocation and digitized and transmitted to a remote location, forexample, via the internet, a LAN, WAN or other intranet, or via astorage medium, such as, for example, CD-ROM, DVD, optical, magnetic,electronic or some other form of currently known or future developedstorage media or transmission method or medium.

For example, where an advertisement is to be placed on the side of atruck or bus by painting or through the application of preprintedmaterials, such as decals, a 2DRI including an outline of the image orof the preprinted materials may be projected onto or superimposed withthe surface, thereby simplifying the task for the operator and allowingthe application or painting to be accomplished quickly and efficiently,and with fewer errors.

In certain applications, for example, where the object being worked onhas a curved surface, a two dimensional representational image may beprojected onto or superimposed with the surface using a holographicoptical element (HOE) to affect the necessary curvature of therepresentational image on the surface. For example, where the paintingor preprinted materials application is to be accomplished on an airplanefuselage, due to the curvature of the fuselage, projecting the 2DRIdirectly onto the fuselage will create undesirable curvature of thepainted image or inaccurate positioning of the preprinted materials.Projecting or superimposing the 2DRI using a HOE will compensate for andcorrect any curvature of the projected or superimposed image due to thecurvature of the surface. Thus, the painted image will be accuratelyrepresented on the surface and the preprinted materials will be properlypositioned.

Where the operator is positioned near the object, an optical combinerinterposed between the operator and the workpiece may be used. Forexample, one type of optical combiner that can be used is a clear glasssheet that can be partially silvered such that some of the light (fromthe first object, the workpiece) passes straight through the combiner toreach the operator's eyes; while the light from a second object (aprojector, or computer display projecting the 2DRI, which may bepositioned at 90 degrees to the workpiece) is reflected off of thecombiner to also reach the operator's eyes. In this way, two independentobjects, the work piece and the 2DRI information can be viewed at thesame time and their virtual position relative to each other can beadjusted so that they can appear to occupy the same space at the sametime—to the operator.

For more complex operations, where a three dimensional image isnecessary to understand the complexities of the operation or theworkpiece, such as detailed or multi-layer component assembly, andworkpiece formation using punches or dies where depth or heightconstraints are critical and/or there are complex curvatures of thesurface, a holographic representational image may be utilized.

As described herein, a holographic representational image (HRI) may becreated using a variety of holographic techniques, including traditionalholographic techniques, and the technique described in U.S. Pat. No.5,748,347 (“V-3D3”). The HRI is created in a manner in which it can beintegrated into the task or analysis that is to be accomplished, in mostinstances with the imagery superimposed with the workpiece. The task oranalysis is then performed in a manner whereby the HRI and theinformation contained therein are used to improve the efficiency andeffectiveness of the task and/or analysis being done, for example, byusing the image to identify or verify the location or position of a partor component.

For example, in a manufacturing process, a two dimensionalrepresentational image (2DRI) or a holographic representational image(HRI) of an object to be manufactured may be created. The number of2DRIs or HRIs used for a manufacturing process may depend, for example,on the number of stages in the manufacturing process, i.e., the numberof workstations or assembly points in the manufacturing line, and/or thenumber of steps or tasks accomplished at each workstation or assemblypoint. For a multiple stage manufacturing process having a single stepor task performed at each of the multiple workstations or assemblypoints, multiple 2DRIs or HRIs may be created using any of the variousknown techniques or techniques described herein, such as, for example,holographic techniques, including traditional holographic techniques andV-3D3. Each 2DRI or HRI created corresponds to a particular stage andstep or stages and/or steps of the manufacturing process. Thus, eachstage or stages of the manufacturing process may be represented by oneor more separate 2DRIs or HRIs that include information that correspondto the step or task, or steps or tasks to be accomplished at that stageor those stages of the manufacturing process.

For a single or multiple stage manufacturing process having multiplesteps or tasks at a single workstation or assembly point, or at multipleworkstations or assembly points, single HRIs, multiple HRIs, compoundHRIs, or any combinations thereof may be used. Alternatively, 2DRIs,multiple 2DRIs or compound 2DRIs may be used, any combinations thereofor any combinations of HRIs and 2DRIs. For each stage of themanufacturing process the type of 2DRI or HRI used may vary depending onthe number and types of steps or tasks to be accomplished. Single HRIsare HRIs having one image contained therein. Multiple HRIs is a group ofsingle HRIs, each representing one of multiple steps performed at asingle stage or multiple stages of the manufacturing process. Likewise,multiple 2DRIs is a group of single 2DRIs, each representing one ofmultiple steps performed at a single stage or multiple stages of themanufacturing process. A compound HRI is a single HRI containingmultiple images. A compound 2DRI is a single 2DRI containing multipleimages. Compound HRIs and 2DRIs will be described in greater detailbelow.

Where a single HRI or 2DRI is used at a stage of the manufacturingprocess, one or more pieces of information may be included in the HRI or2DRI. For example, if there are three components that are to be added tothe product being manufactured at a single stage or assembly point, thelocations of one or more of these three components in the product to bemanufactured may be indicated in a single HRI or 2DRI. The operator mustbe able to distinguish between these three components to accomplish thesteps or tasks effectively.

Where multiple HRIs or 2DRIs are used at each of the stages of themanufacturing process, a single piece of information or multiple piecesof information may be included in each of the HRIs or 2DRIs. Forexample, if there are four components to be added to a product beingmanufactured at a particular stage of the manufacturing process, thelocation of a single component may be indicated in a single HRI or 2DRI,thus requiring four separate HRIs or 2DRIs to provide the informationregarding the four components necessary to complete this stage of themanufacturing process. Using the four HRIs or 2DRIs, this stage of themanufacturing process would be accomplished in four steps.Alternatively, the locations of each of the four components may beindicated in a single HRI or 2DRI, thus requiring a single HRI or 2DRIto accomplish the four steps or tasks included in this stage of themanufacturing process. Another option is to combine HRIs and/or 2DRIshaving single and/or multiple pieces of information to accomplish thesteps or tasks required at the particular stage of the manufacturingprocess. For example, if there are four components to be added to theproduct being manufactured at a particular stage of the manufacturingprocess, the location of a first component may be indicated in a singleHRI or 2DRI and the location of the three other components may beindicated in a separate HRI or 2DRI. This combination of HRIs and/or2DRIs containing different amounts of information would require the useof only two HRIs and/or 2DRIs to accomplish the steps or tasks necessaryfor that stage of the manufacturing process.

A manufacturing process can use multiple 2DRIs or HRIs by switchingbetween each of the single 2DRIs and/or HRIs. This can be donemechanically, such as, for example, by rotating a positioning device inwhich each of the 2DRIs or HRIs is positioned or by moving the object tobe manufactured, or electronically, such as, for example, by utilizingcomputer generated 2DRIs and switching between the computer generated2DRIs. There are many other ways to accomplish such switching.

Using multiple HRIs or 2DRIs may be difficult at times since the amountof space available may be limited, or it may require, for example, thephysical switching or moving of the HRIs or 2DRIs from one location toanother and or repositioning of the object being manufactured. To reduceor eliminate these physical constraints, compound HRIs and/or 2DRIs maybe used. By using a compound HRI or 2DRI the multiple steps or tasksrequired at a particular stage of the manufacturing process may beaccomplished effectively and efficiently. A compound HRI is a single HRIhaving multiple sets of information contained therein. A compound 2DRIis a single 2DRI having multiple sets of information contained therein.Each of the sets of information contained in a compound HRI or 2DRI maybe accessible in a number of different ways, including, for example, byviewing the HRI or 2DRI at different angles or positions, by viewing theHRI or 2DRI under lights of different wavelengths, by viewing the HRI or2DRI from different distances, by placing the HRI or 2DRI at differentdistances from the workpiece, and by illuminating the HRI or 2DRI atdifferent angles. A compound HRI can be created using the V-3D3 processor conventional holography. A compound 2DRI can be created usingconventional photography, printing, or using a computer controlleddisplay, such as, for example, a liquid crystal display (LCD), a plasmadisplay, a light emitting diode (LED) display, spacial light modulator,fiber optics, or any other display that can either project orsuperimpose an image using a light or other visible energy source, orallow light to pass through for projecting or superimposing an image ofthe displayed information.

A compound 2DRI using a single transparency may be created in a numberof different ways. For example, the two dimensional information mayfirst be scan analyzed into the necessary subcomponents, for example,vertical or horizontal stripes. These sub-components are correlated withone or more lenticulated lens elements. The sub-components from a firstset of two dimensional information can then be interposed with thesub-components from a second set (or more sets) of two dimensionalinformation in a fixed relationship. The resulting series ofsub-components, for example, stripes, are situated in a predeterminedspatial relationship to the lenticulated lens/filter, for example, bylaminating or printing the stripes in register with the lenticulatedlens/filter. The resulting compound 2DRI will present each of the setsof two-dimensional information, for example, images, to the projectionoptics (which can be similar to that from a slide projector) when thelenticulated side of the 2DRI is illuminated from a particular off-axisangle. Alternatively, the sub-components may be correlated with one ormore polarizing filter. Each set of 2-D information will be presentedupon illumination by light having the corresponding polarization.

A compound 2DRI using a single transparency may also be created usingcolor separation techniques, for example, by providing therepresentation of each set of two dimensional information in a differentcolor or corresponding to a different wavelength of light. Theprojection optics would then present each of the sets of two-dimensionalinformation using a light source of a different color or wavelength, forexample, by using colored filters.

After creation of each 2DRI and/or HRI to be used in the manufacturingprocess, the 2DRIs and/or HRIs must be physically arranged so that theinformation contained in the 2DRIs and/or HRIs, for example, informationregarding the location of a specific component to be inserted during themanufacturing process or the location, position, or shape of a cut orbend, may be utilized by the operators at the various stages of themanufacturing process. This may be accomplished by removably positioningeach of the 2DRIs and/or HRIs at their corresponding stages of themanufacturing process in fixed or adjustable relation to the location ofthe object to be manufactured. The object to be manufactured is placedin a predetermined position, which may be fixed or adjustable by theoperator. The 2DRI or HRI is then coupled to, or positioned on or in amounting or positioning device, which is either fixed or adjustable, atsome predetermined or adjustable distance from the object to bemanufactured. A light source is then positioned in such a way that theinformation contained in the 2DRI or HRI may be utilized by an operator,for example, by projecting or superimposing the 2DRI or HRI informationonto or with the object to be manufactured, directly or indirectly, suchthat the 2DRI or HRI information and the object are superimposed, i.e.,they overlap, and for the HRI they are substantially in register or nearregister in all three dimensions, including depth. These steps may berepeated for each stage of and step in the manufacturing process, or foras many stages and/or steps as desired. As the object moves through thestages of the manufacturing process, at each stage and for each step the2DRI or HRI information is available to the operator, for example, bybeing projected onto or superimposed with the object to be manufactured,thereby clearly indicating to the operator the required task, forexample, the location of the component to be inserted or the location,position or shape of a cut or bend.

Thus, the manufacturing process will include a new source of informationfor the operators that will provide improved guidance, for example,through the use of a Two or Three Dimensional Visual Template (2DVT or3DVT) type of representational image presented in superimposition withthe object to be manufactured. This 2DVT or 3DVT representational imagecan be used by the operator to identify the task to be performed, forexample, the location of the component on the object being manufacturedor the position or shape of a cut or bend. The 2DVT utilizes theprojected or superimposed image to capture in 2-D the visual image ofits subject, and the 3DVT utilizes the unique properties of the hologramto capture in full 3-D, including depth, the visual image of itssubject.

For a manufacturing process having multiple steps or tasks at eachstage, for example, in the construction of a digital clock, where thecircuit board is constructed at one stage, in multiple steps, thedisplay at another stage, also in multiple steps, etc., a single 2DRI orHRI may be used for a single step or task, thereby requiring multiple2DRIs or HRIs to be used to accomplish the necessary steps or tasks at aparticular stage of the manufacturing process. To accomplish the varioussteps, the operator is provided access to the information contained ineach 2DRI or HRI and is able to cycle or switch between each of themultiple 2DRIs and/or HRIs that are utilized. This may be accomplishedby successively displaying the various 2DRIs and/or HRIs and allowingthe operator to view the information contained in each 2DRI and HRI andaccomplish the task to which such 2DRI or HRI corresponds, beforedisplaying the next 2DRI and/or HRI. For example, by using a mounting orpositioning device with a rotating portion, the various 2DRIs and/orHRIs may be coupled to, or positioned on or in a mounting or positioningdevice, which is either fixed or adjustable, at some predetermined oradjustable distance from the object to be manufactured, in planarfashion. As the rotating portion rotates within a plane substantiallyparallel to the plane in which the object to be manufactured ispositioned, the various 2DRIs and HRIs can be moved into a position fromwhich the information from the 2DRI or HRI can become substantially inregister with the object to be manufactured. A light source ispositioned as described above in a manner that will allow theinformation contained in the 2DRI or HRI to be utilized by an operator,for example, by projecting or superimposing the 2DRI or HRI informationonto or with the object to be manufactured such that the projected orsuperimposed 2DRI or HRI information and the object are substantially inregister or near register. When the operator has completed the requiredtask, the apparatus may be rotated so as to move the next 2DRI or HRIinto the proper position for the next operation.

When a compound HRI is used to provide the representational images to beused during the manufacturing process, the different sets of informationcontained in the compound HRI may be accessible by viewing the HRI atdifferent angles or from different positions. This may be accomplishedby placing the HRI in an adjustable positioning device. The positioningdevice may be adjustable by fixed or variable increments. After thefirst set of information is accessed by the operator throughillumination of the HRI, as described above, the HRI may then berepositioned, for example, by moving the HRI horizontally, vertically,diagonally or a combination thereof. The HRI is then illuminated againto allow the operator to access the next set of information. Thisprocess is continued until all of the information contained in the HRInecessary for the operator to accomplish each required step or task isaccessed. Similarly, the different sets of information contained in acompound 2DRI may be accessible by illuminating the 2DRI from differentangles either by moving the 2DRI or the light source as described above.This may be accomplished by placing the 2DRI in an adjustablepositioning device as described above for the HRI.

Accessing the various sets of information in a compound 2DRI or HRI mayalso be accomplished by viewing the 2DRI or HRI under lights ofdifferent wavelengths. First the 2DRI or HRI is placed in a positioningdevice, as described above. The 2DRI or HRI is then illuminated using alight source of a first predetermined wavelength. This allows theoperator to access the first set of information contained in the 2DRI orHRI. After the operator has completed the step for which the first setof information was accessed, the 2DRI or HRI is illuminated using alight source of a second predetermined wavelength. This process iscontinued until all of the information contained in the 2DRI or HRInecessary for the operator to accomplish each required step or task isaccessed.

Accessing the various sets of information in a compound 2DRI or HRI mayalso be accomplished by viewing the 2DRI or HRI from differentdistances. First the HRI is placed in a positioning device, as describedabove. The 2DRI or HRI is adjusted to a position at a firstpredetermined distance from the plane of the object to be manufactured,i.e., a first predetermined focal point. This allows the operator toaccess the first set of information contained in the 2DRI or HRI. Afterthe operator has accessed the first set of information throughillumination of the 2DRI or HRI, as described above, the 2DRI or HRI isthen repositioned, for example, by moving the 2DRI or HRI toward or awayfrom the object to be manufactured, i.e., to a second predeterminedfocal point. The 2DRI or HRI is then illuminated again to allow theoperator to access the next set of information. This process iscontinued until all of the information contained in the 2DRI or HRInecessary for the operator to accomplish each required step or task isaccessed. Alternatively, the various sets of information contained in acompound 2DRI or HRI may be accessible by viewing the 2DRI or HRI fromdifferent distances from the plane of the 2DRI or HRI, or by moving theobject to be manufactured toward or away from the 2DRI or HRI. Toprovide for this type of viewing, the 2DRI and HRI must be created withthe different sets of information at different distances or focal pointsfrom its surface, with a corresponding change in the light sourcedistance or projection optics.

Accessing the various sets of information in a compound 2DRI or HRI mayalso be accomplished by illuminating the 2DRI or HRI at differentangles. This may be accomplished by placing the 2DRI or HRI in apositioning device. The light source used to illuminate the 2DRI or HRImay be adjustable by fixed or variable increments or multiple adjustablelight sources may be used. The light source is placed in a firstpredetermined position, or a first light source (located at a firstpredetermined position) may be used. The 2DRI or HRI is then illuminatedallowing the operator to access the first set of information locatedtherein. After the operator accesses the first set of information, thelight source may then be repositioned, for example, by moving the lightsource horizontally, vertically, diagonally or a combination thereof, ora second light source may be used. Alternatively, the 2DRI or HRI may berepositioned. The 2DRI or HRI is then illuminated again to allow theoperator to access the next set of information. This process iscontinued until all of the information contained in the 2DRI or HRInecessary for the operator to accomplish each required step or task isaccessed.

Utilizing a compound 2DRI or HRI, a multiple stage manufacturing processmay be consolidated into a single physical stage and/or numerous stepsof a manufacturing process may be consolidated so as to be accomplishedby a single operator, thereby reducing the physical size of themanufacturing line and reducing the number of operators. The timerequired for training may also be reduced, for example, because all ofthe tasks to be accomplished will be viewable as a projected orsuperimposed image on the workpiece, which will serve as a template forthe operator.

The quality assurance (QA) and quality control (QC) processes may alsobe enhanced and improved using 2DRIs and/or HRIs. For example, insteadof a manual QA/QC review by an inspector who must identify everycomponent in the object being manufactured and verify the accurateplacement of each component, usually by reference to a schematic ormodel, the QA/QC review can be made using a 2DRI or HRI. Using a 2DRI orHRI allows the comparison of the finished workpiece to the model orschematic to be accomplished using a projected or superimposed image ofthe desired final product with all of the necessary components. This canbe achieved by utilizing multiple 2DRIs and/or HRIs with each componentrepresented by an individual 2DRI or HRI, single 2DRIs or HRIs eachhaving a plurality of components for projection onto or superimpositionwith the manufactured object, a compound 2DRI or HRI with each of thesets of information, e.g., components, viewed using one of thetechniques described above, or any combination of the above. Since eachof the manufactured objects can be QA/QC inspected using a 2DRI or HRIor 2DRIs or HRIs, there can be 100% QA/QC for every object manufactured.

Using 2DRIs and/or HRIs, the QA/QC inspection process may becomepartially or even fully automated. After all of the manufacturing stepshave been completed, the finished product may be sent to anotherstation. At this station, the QA/QC review is begun. A 2DRI or HRI ispositioned in fixed or adjustable relation to the finished product. Theinformation contained in the 2DRI or HRI is projected or superimposedonto the finished product. An inspector or an automated detectiondevice, such as, for example, a machine vision system, or both, view thefinished product with the projected or superimposed 2DRI or HRIinformation and make a determination as to whether the finished producthas passed inspection. If the finished product does not pass inspection,the inspector and/or the automated detection device will then be able toidentify the specific items that caused the failed inspection. Thefinished product may then be sent back to the corresponding location(s)in the manufacturing process to correct what caused the failedinspection. A compound 2DRI or HRI can be used in place of a single 2DRIor HRI, respectively, by using one of the processes described above toproject or superimpose, in succession, the various sets of informationcontained in the compound 2DRI or HRI onto the finished product for aQA/QC inspection. For example, each of the sub-assembly images could besuperimposed on the work piece in quick succession to confirm that eachelement that was to be added in the production line is in place and inthe correct location. Such inspection could identify faulty piecesinstantly, as well as which workstation would need to do the revision onthe piece to achieve Zero Defects.

For example, the 2DRI and/or HRI can be used in conjunction with amachine vision system to effectively take over a portion of the QA/QCinspection. In place of a human inspector, the machine vision systemwould detect errors in a workpiece by a comparison of the workpiece tothe image of the correct object as projected or superimposed by the 2DRIor HRI. The machine vision system could then identify whether any errorsexist and any necessary revisions. The machine vision system could alsocompile information about the manufacturing assembly line, for example,relating to which steps and/or workers exhibit above average or abovenormal error rates.

The language free aspect is also significant considering the expandinguse of a multinational workforce and overseas manufacturing, and thecost savings associated with centralizing any training operations.Additionally, with the use of low cost labor, there is often a languageissue since quite often English is the first the language of themanagement and training department, while the first language of suchlabor force is not English.

Finally, many products are currently designed to minimize the use ofhandwork, however this comes at a high robotic cost. Not only must theassemblies be designed to match the capabilities of the robotic systems(sometimes requiring sub-optimal design of the assembly), but the costto the manufacturer for model changes can be quite high because of thecost of reprogramming the robots that do the assemblies. Using one ormore of these embodiments will provide companies with the ability toachieve very near ‘robotic’ manufacturing quality (very low rejectrates) and provide the ability to have workers trained efficiently andcost effectively and adapt very quickly to changes in the assembly line,new assembly requirements, and different products. It will providecompanies with a way to decrease the overall cost of manufacture andQA/QC, while maintaining the same or even higher quality standards.

BRIEF DESCRIPTION OF THE DRAWINGS

FIGS. 1( a) and 1(b) show a top view and a side view (along a planedefined by points a-a), respectively, of a positioning and setupoperation according to an embodiment.

FIG. 2 shows a flow diagram of an assembly operation according to anembodiment.

FIGS. 3( a) and 3(b) show positions for a sub-frame according to anembodiment.

FIG. 4 shows a quality control/quality assurance procedure according toan embodiment.

FIG. 5 shows a quality control/quality assurance procedure utilizing acompound HRI, according to an embodiment.

FIGS. 6( a) and 6(b) show a top view and a side view (along a planedefined by points a-a), respectively, of a positioning and setupoperation according to an embodiment.

FIG. 7 shows a flow diagram of an assembly operation according to anembodiment.

FIGS. 8( a) and 8(b) show positions for the sub-frame when a compoundHRI is used according to an embodiment.

DETAILED DESCRIPTION

Referring first to FIGS. 1( a) and 1(b), there is shown a top view and aside view (along a plane defined by points a-a), respectively, of apositioning and setup operation according to an embodiment. FIGS. 1( a)and 1(b) show a manufacturing assembly line 1 where an operator 3 at astep 5 of the manufacturing process is working on the assembly of aproduct. The product is assembled using a sub-frame 7 and a number ofcomponents. At this step 5 of the manufacturing process, the operator 3is responsible for the insertion and connection of a component 9 ontothe sub-frame 7. Before any assembly is performed, an HRI 11 is createdusing traditional holographic techniques. Alternatively, HRI 11 could bea 2DRI.

The information contained in the HRI 11 includes a representationalimage of the sub-frame 7 having the same shape, dimensions and physicalstructure as the actual sub-frame 7 when viewed in its proper positionof assembly at step 5 of the manufacturing process. The HRI 11 alsoincludes a representational image of the component 9 that is to be addedduring step 5 in its proper position of assembly. Once created, the HRI11 is positioned on or in an adjustable frame 13 that is coupled to thework area adjacent to the operator 3. The adjustable frame 13 may bepositioned using a first positioning device 15 or a second positioningdevice 17, which may be, for example, a rotating knob or lever connectedto a mechanical gear or mechanism, a mechanical assembly, or anelectronic switch connected to a motor. Alternatively, both the firstpositioning device 15 and the second positioning device 17 may be usedto position the adjustable frame 13 or additional positioning devicesmay be used. The sub-frame 7 is positioned in correlation to the HRI 11.Prior to adjusting the position of the adjustable frame 13, the HRI 11is illuminated by light source 21. Upon illumination of the HRI 11 bylight source 21, the representational image stored in the HRI 11,including the holographic 3-dimensional image of the sub-frame 7 and thecomponent 9, is superimposed with the actual subframe 7. Once therepresentational image contained in the HRI 11 is superimposed with theactual sub-frame 7, the adjustable frame 13 is then adjusted using oneor both of the first positioning device 15 and the second positioningdevice 17, until the representational image presented by the HRI 11 issubstantially in or near register with and appears to overlap the actualsub-frame 7. The first positioning device 15 and the second positioningdevice 17 may then be fixed in position. Alternatively, the sub-frame 7may be adjusted until the representational image presented by the HRI 11is substantially in or near register with and appears to overlap theactual sub-frame 7.

Referring now to FIG. 2, there is shown a flow diagram of an assemblyoperation according to an embodiment. Once the HRI 11 is adjusted, themanufacturing assembly line may be operated. In a first step 22, eachsub-frame 7 is moved into position at step 5 of the manufacturingassembly line 1. This is followed by step 23, where the HRI 11 isilluminated by the light source 21. Upon illumination of the HRI 11 bythe light source 21, the representational image stored in the HRI 11,including a holographic 3-dimensional image of the sub-frame 7 and thecomponent 9, is superimposed with the actual sub-frame 7. Once therepresentational image contained in the HRI 11 is superimposed with theactual sub-frame 7, in a next step 24, the operator 3 performs anassembly operation by placing the component 9 in the locationcorresponding to that indicated by the presented image of the component9 with sub-frame 7. When assembly step 24 is completed, in a step 25,the sub-frame is moved to the next step of the manufacturing assemblyline 1, where further assembly may be accomplished. The same operationsas described above may be implemented for the next and any other stagesof the manufacturing assembly line 1 as desired.

Referring now to FIGS. 3( a) and 3(b), there is shown a position for thesub-frame according to an embodiment. Alternatively, as shown in FIG. 3(a), the sub-frame 7 may be positioned such that side 30, which islocated furthest from the operator 3, is raised with respect to side 32,which is located nearest the operator 3. Thus, the sub-frame 7 may bepositioned on a plane parallel to the plane on which the HRI 11 ispositioned. Alternatively, as shown in FIG. 3( b), the sub-frame 7 maybe positioned in or on an adjustable device 33, whereby the position ofthe sub-frame 7 may be adjusted relative to the HRI 11 using one or moreof positioning devices 35. When all of the steps of the manufacturingassembly line 1 are initialized, the manufacturing assembly line 1 maybe operated.

Referring now to FIG. 4, there is shown a quality control/qualityassurance procedure according to an embodiment. Upon completion of theassembly of the components on the sub-frame 7, the sub-frame 7 is movedto a quality control location 40. At quality control location 40 aplurality of HRIs 42, corresponding to the number of steps in themanufacturing assembly line 1, are adjustably (or fixedly) positioned ina fixed or adjustable frame 44. Alternatively, a compound HRI containingall or some of the information corresponding to the steps of themanufacturing assembly line 1, or any number of HRIs may be used. EachHRI 42 includes a representational image of a single component or aplurality of components in its or their proper location(s) on thesub-frame 7. The sub-frame 7, which is located in an assessment position46, is positioned in correlation to HRI 42(a). The frame 44 may beadjusted so that upon illumination of the HRI 42(a) by light source 48,the representational image stored in the HRI 42(a), including theholographic 3-dimensional image of the sub-frame 7 and a singlecomponent or plurality of components, is superimposed with the actualsub-frame 7. Once the representational image contained in the HRI 42(a)is superimposed with the actual sub-frame 7, the frame 44 is thenadjusted, if necessary, using one or both of a first positioning device50 and a second positioning device 52, until the holographicrepresentational image presented by the HRI 42(a) is substantially in ornear register with and appears to overlap the actual sub-frame 7. Thefirst positioning device 50 and the second positioning device 52 maythen be fixed in position. The HRIs 42 are then alternated such thateach is positioned in correlation to sub-frame 7 in assessment position46, illuminated by light source 48 and assessed to determine if assemblywas performed properly. Alternatively a 2DRI, multiple 2DRIs or acompound 2DRI may be used in place of HRIs 42.

Referring now to FIG. 5, there is shown a quality control/qualityassurance procedure utilizing a compound HRI, according to anembodiment. In place of the HRIs 42, alternatively, a compound HRI 60may be used which contains some or all of the information contained ineach or some of the HRIs 42. The compound HRI 60 is adjustably (orfixedly) positioned in a fixed or adjustable frame 62. The compound HRI60 includes a plurality of representational images, each of a singlecomponent or a plurality of components in its or their properlocation(s) on the sub-frame 7. The sub-frame 7, which is located in anassessment position 46, is positioned in correlation to compound HRI 60.The frame 62 may be adjusted so that upon illumination of the compoundHRI 60 by a first light source 64, a first representational image ofthree-dimensional information stored in the compound HRI 60, includingthe holographic three-dimensional image of the sub-frame 7 and a singlecomponent or plurality of components, is superimposed with the actualsub-frame 7. Once the first representational image contained in thecompound HRI 60 is superimposed with the actual sub-frame 7, the frame62 is then adjusted, if necessary, using one or both of a firstpositioning device 66 and a second positioning device 68, until thefirst representational image presented by the compound HRI 60 issubstantially in or near register with and appears to overlap the actualsub-frame 7. The first positioning device 66 and the second positioningdevice 68 may then be fixed in position. Alternatively, a compound 2DRImay be used in place of compound HRI 60.

When assessing the assembly of a product, the sub-frame 7 is positionedin assessment position 46 the compound HRI 60 in frame 62 is illuminatedusing the first light source 64 to present a first representationalimage of three-dimensional information stored in the compound HRI 60,including the holographic three-dimensional image of the sub-frame 7 anda single component or plurality of components, with the sub-frame 7 forassessing the assembly of a first component 70, or any number ofcomponents. The assessment may be accomplished by human review or byreview using an automated detection device, such as, for example, amachine vision system. The first assessment includes a comparison of thesuperimposed first representational image to the first component 70 onthe sub-frame 7 to determine if there is uniformity.

The compound HRI 60 is then illuminated by a second light source 72. Asecond representational image of three-dimensional information stored inthe compound HRI 60, including the holographic three-dimensional imageof the sub-frame 7 and a single component or plurality of components, issuperimposed with the actual sub-frame 7 for assessing the assembly of asecond component 74, or any number of components. The assessment may beaccomplished by human review or by review using an automated detectiondevice. The second assessment includes a comparison of the superimposedsecond representational image to the second component 74 on thesub-frame 7 to determine if there is uniformity. These steps arerepeated for as many representational images as desired, for example, asmany as are included in the compound HRI 60 or in the product, each timeusing a different light source, and until an assessment has been made ofthe assembly of all or a selected number of the components. For eachcomponent or each group of components a new light source is used toilluminate the compound HRI 60 and present a new representational imageof three-dimensional information stored on the compound HRI 60.

Referring now to FIGS. 6( a) and 6(b) (which for all identicalcomponents utilize the numbering from FIGS. 1( a) and 1(b)), there isshown a top view and a side view (along a plane defined by points a-a),respectively, of a positioning and setup operation according to anembodiment. FIGS. 6( a) and 6(b) show a manufacturing assembly line 1,where an operator 3 at a station 80 of the manufacturing process isworking on the assembly of a product. The product is assembled using asub-frame 7 and a number of components. At this station 80 of themanufacturing process, the operator 3 is responsible for the performanceof a number of steps of the assembly, including the insertion andconnection of a plurality of components onto the sub-frame 7.

Before any assembly is performed, a compound HRI 82 is created usingV-3D3 techniques as described in U.S. Pat. No. 5,748,347. Theinformation contained in the compound HRI 82 includes a plurality ofrepresentational images of the sub-frame 7 having the same shape,dimensions and physical structure as the actual sub-frame 7 when viewedin its proper position of assembly at station 80 of the manufacturingprocess. The compound HRI 82 also includes representational images ofthe components that are to be added during a plurality of stepsperformed at station 80 in their proper positions of assembly. Oncecreated, the compound HRI 82 is positioned on or in an adjustable frame13 that is coupled to the work area adjacent to the operator 3. Theadjustable frame 13 may be positioned using a first positioning device15 or a second positioning device 17, which may be, for example, arotating knob or a lever connected to a mechanical gear or mechanism, amechanical assembly, or an electronic switch connected to a motor.Alternatively, both the first positioning device 15 and the secondpositioning device 17 may be used to position the adjustable frame 13.The sub-frame 7 is positioned in correlation to the compound HRI 82.

Prior to adjusting the position of the adjustable frame 13, the compoundHRI 82 is illuminated by a first light source 84. Upon illumination ofthe compound HRI 82 by the first light source 84, a firstrepresentational image stored in the compound HRI 82, including aholographic 3-dimensional image of the sub-frame 7 and a first component86, is superimposed with the actual sub-frame 7. Once the firstrepresentational image contained in the compound HRI 82 is superimposedwith the actual sub-frame 7, the adjustable frame 13 is then adjustedusing one or both of the first positioning device 15 and the secondpositioning device 17, until the first representational image presentedby the compound HRI 82 is substantially in or near register with andappears to overlap the actual sub-frame 7. The first positioning device15 and the second positioning device 17 may then be fixed in position.Alternatively, a compound 2DRI may be used in place of compound HRI 82.

Referring now to FIG. 7, there is shown a flow diagram of an assemblyoperation according to an embodiment. Once the compound HRI 82 isadjusted, the manufacturing assembly line may be operated. In a firststep 93, each sub-frame 7 is moved into position at station 80. In thenext step 94, the compound HRI 82 is illuminated by a first light source84. Upon illumination of the compound HRI 82 by the first light source84, a first representational image stored in the compound HRI 82,including a holographic 3-dimensional image of the sub-frame 7 and afirst component 86, is superimposed with the actual sub-frame 7. Oncethe first representational image contained in the compound HRI 82 issuperimposed with the actual sub-frame 7, in a next step 95, theoperator 3 performs a first assembly operation by placing the firstcomponent 86 in the location corresponding to that indicated by thesuperimposed image of the first component 86 with sub-frame 7. Whenassembly step 95 is completed, in step 96 the first light source 84 isremoved and compound HRI 82 is then illuminated by a second light source88. Upon illumination of the compound HRI 82 by the second light source88, a second representational image stored in the compound HRI 82,including a holographic 3-dimensional image of the sub-frame 7 and asecond component 90, is superimposed with the actual sub-frame 7. Oncethe second representational image contained in the compound HRI 82 issuperimposed with the actual sub-frame 7, in step 97 the operator 3performs a second assembly operation by placing the second component 90in the location corresponding to that indicated by the superimposedimage of the second component 90 with sub-frame 7. These operations arerepeated for as many steps as are performed at station 80, or for asmany representational images as are included in the compound HRI 82. Foreach representational image a different light source is used toilluminate the compound HRI 82. Upon completion of the station 80assembly steps, in step 98, the sub-frame 7 is moved to a new station onthe manufacturing assembly line 1 and these operations may be repeated.

The QC/QA process may be performed as described with reference to FIG.5, or alternatively, the QC/QA process may be integrated with theassembly process as described above with respect to FIG. 6. Anintegrated QC/QA process may be accomplished using the representationalimages stored in the compound HRI 82. In addition to the informationincluded in each representational image contained in the compound HRI82, i.e., a holographic 3-dimensional image of the sub-frame 7 and thefirst component 86, as described above, the compound HRI 82 must alsoinclude the component or components from the prior assembly step orsteps of the manufacturing assembly line 1. Each step of themanufacturing assembly line 1 can check the assembly from the prior stepor from any number of prior steps.

For example, if each step incorporates a QC/QA inspection of theassembly performed at the prior step, then the representational imagefor the second step in the compound HRI 82 would include a holographic3-dimensional image of the sub-frame 7, the first component 86, and thesecond component 90. The second component 90 would be included for thepurposes of assembly and the first component 86 would be included forthe purposes of QC/QA assessment. The operator 3 viewing the actualsub-frame 7 would be able to assess whether the assembly from the priorstep was performed properly by comparing the 3-dimensional image offirst component 86 to the actual first component 86. If they aresubstantially superimposed one with the other, then the prior assemblystep has passed inspection, and if not then the sub-frame 7 may be sentfor rework.

Alternatively, if each step of the manufacturing assembly line 1incorporates a QC/QA inspection of the assembly performed at all priorsteps, then the representational image for each step in the compound HRI82 would include a holographic 3-dimensional image of the sub-frame 7,the component currently being assembled, and all components from allprior assembly steps. Thus, if assembly step 95 is the third assemblystep of the manufacturing assembly line 1, the representational imagecontained in the compound HRI 82 for assembly step 95 would include aholographic 3-dimensional image of the sub-frame 7, component 86 and thecomponents assembled during the two prior steps. The operator 3 viewingthe actual sub-frame 7 would be able to assess whether the assembly fromthe prior step was performed properly by comparing the 3-dimensionalimage of component 86 to the actual component 86. If they aresubstantially superimposed one with the other, then the prior assemblystep has passed inspection, and if not then the sub-frame 7 may be sentfor rework.

This type of integrated QC/QA process may also be applied to theassembly process as described with respect to FIG. 1. For example, ifstep 5 was the fourth step of the manufacturing assembly line 1, therepresentational image contained in HRI 11 would include a 3-dimensionalimage of the sub-frame 7, component 9 and the components assembledduring the three prior steps. The operator 3 viewing the actualsub-frame 7 would be able to assess whether the assembly from the priorstep was performed properly by comparing the 3-dimensional image ofcomponent 9 to the actual component 9. If they are substantiallysuperimposed one with the other, then the prior assembly step has passedinspection, and if not then the sub-frame 7 may be sent for rework.

Referring now to FIGS. 8( a) and 8(b), there are shown positions for thesub-frame when a compound HRI is used according to an embodiment.Alternatively, as shown in FIG. 8( a), the sub-frame 7 may be positionedsuch that side 30, which is located furthest from the operator 3, israised with respect to side 32, which is located nearest the operator 3.Thus, the sub-frame 7 may be positioned on a plane parallel to the planeon which the compound HRI 82 is positioned. Alternatively, as shown inFIG. 8( b), the sub-frame 7 may be positioned in or on an adjustabledevice 33, whereby the position of the sub-frame 7 may be adjusted, forexample, using one or more of positioning devices 35, relative to thecompound HRI 82. When all of the steps of the manufacturing assemblyline 1 are completed, the manufacturing assembly line 1 may be operated.

1. A method for performing quality control on a processed object, duringor after processing or manufacturing, using a representational image,comprising: positioning the processed object at a predetermined positionrelative to the representational image; illuminating therepresentational image; presenting information from the representationalimage to an operator as a result of the illuminating; at least one ofcontrolling the illuminating of the representational image andpresenting of information from the representational image to anoperator, using an electronic processing arrangement; and at least oneof accepting the processed object and rejecting the processed objectusing the information obtained from the representational image.
 2. Themethod of claim 1, wherein the presenting is accomplished bysuperimposing the representational image with the processed object. 3.The method of claim 2, wherein the superimposing is accomplished usingan holographic optical element.
 4. The method of claim 2, wherein thesuperimposing includes at least one of projecting the representationalimage onto the processed object and holographically displaying therepresentational image in relation to the processed object.
 5. Themethod of claim 1, wherein the at least one of illuminating therepresentational image and presenting of information from therepresentational image to an operator is accomplished using at least oneof a slide projector, a computer image projector, an overhead projector,a high intensity computer display, an optical combiner, a prismaticscreen, a splitter, a liquid crystal display, a plasma display, a lightemitting diode display, a spatial light modulator, and fiber optics. 6.The method of claim 1, wherein the representational image is at leastone of a two-dimensional image, a three-dimensional image and aholographic image.
 7. The method of claim 6, wherein the two-dimensionalimage is a computer generated image.
 8. The method of claim 6, whereinthe two-dimensional image is presented by at least one of a liquidcrystal display, a plasma display and a light emitting diode display. 9.The method of claim 1, wherein the processed object is positioned on amovable surface.
 10. The method of claim 1, wherein the processed objectis positioned at a distance from the representational imagesubstantially equal to the focal distance of the representational image.11. The method of claim 1, wherein the representational image isilluminated using at least one of white light and light of apredetermined wavelength.
 12. The method of claim 1, wherein thepresenting is accomplished by reflecting the representational image ontothe processed object.
 13. The method of claim 1, wherein information ispresented from the representational image to the operator by at leastone of reflection and transmission.
 14. The method of claim 1, whereinthe at least one of controlling the illuminating of the representationalimage and presenting of information from the representational image toan operator is accomplished by adjusting at least one of the angle ofthe representational image, the position of the representational imageand the wavelength of the light used to illuminate the representationalimage.
 15. The method of claim 1, wherein the at least one ofcontrolling the illuminating of the representational image andpresenting of information from the representational image to an operatoris accomplished by using a filter.
 16. The method of claim 15, whereinthe filter is at least one of a polarizing filter, a lenticulated lensand a color filter.
 17. The method of claim 1, wherein the operator isan automated detection device.
 18. The method of claim 17, wherein theautomated detection device is a machine vision system.
 19. The method ofclaim 1, further comprising: comparing the information obtained from therepresentational image to the processed object; and accepting orrejecting the processed object based on the comparison.
 20. A system forperforming quality control on a processed object, during or afterprocessing or manufacturing, using at least one representational image,comprising: a repository of the at least one representational imagepositioned at a predetermined distance from the processed object; anilluminator providing an illumination of at least one predeterminedwavelength to the repository of the at least one representational image,whereby information from the at least one representational image istransmitted to an operator; and an operator positioned at apredetermined distance from the repository of the at least onerepresentational image, the operator analyzing the information from theat least one representational image and at least one of accepting theprocessed object and rejecting the processed object based on acomparison of the information from the at least one representationalimage to the processed object.
 21. The system of claim 20, wherein theoperator is a human being.
 22. The system of claim 20, wherein theoperator is an automated detection system.
 23. The system of claim 20,wherein the automated detection system is a machine vision system.